Decentralized Resource Allocation in Application Layer Networks

Application-layer networks (ALN) are software architectures that allow the provisioning of services requiring a huge amount of resources by connecting large numbers of individual computers. The ALN simulation project CATNET evaluates a decentralized mechanism for resource allocation in ALN, which is based on the economic paradigm of the Catallaxy, against a centralized mechanism using an arbitrator object. In both versions, software agents buy and sell network services and resources to and from each other. The economic model is based on self-interested maximization of utility and self-interested cooperation between agents. This article describes the design of money and message flows for centralized and decentralized coordination in both versions and shows preliminary results. 1. Allocation of Resources in Application Layer Networks Application-layer networks (ALN) are software architectures that coordinate the provisioning of services requiring a huge amount of resources by connecting large numbers of individual computers. Such global Internetbased networks, like today’s Grids [2] and Peer-to-PeerComputing [14], take advantage of such infrastructures with applications like multicast services for global audiences, storage repositories of peta-scale data sets, or parallel computing applications requiring teraflops of processing power. Such applications are executed in multiple resource locations distributed throughout the Internet, coordinated on the application layer using a dedicated network, the ALN. An ALN scenario would be the distributed provisioning of web services for Adobe’s Acrobat (for creating PDF files). Here, word-processor client programs would transparently address the nearest/ cheapest Acrobat service instance in order to create PDF files. The overall objective of the ALN would be (a) to always provide access to some Acrobat service instance, such that a minimum number of service demands have to be rejected, and (b) to optimize network parameters such as provisioning and transmission costs. This paper assumes that the future development of these applications will lead to clients paying for the access to a service and the corresponding onor offline exchange of payment; the individual goal of a client would become to access a service cheaply, while services may try to maximize income. In order to keep an ALN operational, service control and resource allocation mechanisms are required. Their basic purpose would be to match service supply and demand, in the likely case of multiple, redundant service instances, to meet those objectives. The simple service discovery mechanisms available today in decentralized networks (e.g. Jini [19]) seldom provide such functionality, as the case of redundant service instances is yet rare. However, a realization of these mechanisms by employing a centralized coordinator instance (auctioneer, arbitrator, dispatcher, scheduler, manager), like e.g. in GLOBUS [8] or CONDOR-G [9], has several drawbacks. First, ALN and the underlying networks are very dynamic and fast changing systems: service demands and nodes connectivity changes are frequent, and new different services are created and composed continuously. Information collected from the network is considered to be outdated when it reaches the coordinator; any solution computed on the basis of this information tries to optimize a past and inconsistent state of the network. Dynamic ALN need a continuous, real-time coordination mechanism, which reflects the changes in the environment. A second related property is that the coordinator should have global knowledge on the state of the network. This is mostly achieved by calculating the time steps such that actual status information from all nodes arrives safely at the coordination instance. However, if the diameter of the network grows, this approach leads to long latency times for the nodes. Third, a centralized coordinator is part of the problem that decentralized ALN are trying to solve: As bids and offers have to route through the network to the single instance which collects global knowledge and computes the resource allocation, the distribution and deployment of services throughout the network is counteracted. This is currently not a problem as the control information is small compared to the allocation data itself, but may increase when the principle is applied to more and more application areas. These drawbacks lead to the search for a truly decentralized coordination concept which is able to allocate services and resources in real -time without a dedicated coordinator instance. This concept should on one hand be able to cope with technical shortcomings like varying amounts of memory and disk space, internet connection speed and sporadic appearance and disappearance of the services. On the other hand, it is desirable that the network as a whole shows optimized behavior with regard to low overhead communication, short computation times, and economical resource allocation. Recent research in Grid computing has also recognized the value of price generation and negotiation, and in general investigates economic models for trading resources and services and the regulation of supply and demand of resources in an increasingly large-scale and complex Grid environment. A general overview on resource management and scheduling in Grids is given in [4]. However, each Grid project is differently designed, and it is not po ssible to compare different allocation mechanisms within the same network without changing the fundamentals. In the remainder of this article, we first introduce a decentralized economic concept for coordination, the Catallaxy, and describe the CATNET project. The following section compares money and message flows in the application-layer network economic model, both with a centralized (baseline) and a decentralized implementation. Next we describe how the experiments are conducted in both cases. The article closes with some preliminary experimental results and an outlook to further research. 2. The Catallaxy Paradigm and the CATNET